AlaskaLinuxUser's Scratchpad

Commit thy works unto the LORD, and thy thoughts shall be established. - Proverbs 16:3

Running a local AI model with Ollama and AI pitfalls

20260826.png

To be upfront and honest, I'm not a very big fan of AI. I litterally think of it as a machine that predicts the next word, pixel, image, code, thing, based on statistics. One of the dangers of such a machine is that it does so in a "life-like" manner, which can sound very natural, while actually giving you completely false information. I know that there are plenty of people who do like using AI, and I'm not faulting them, but I don't think it is a very good idea for me personally. However, in light of all of the above, while I don't like to use AI, I do need to understand how it works, and how to use it should the need arise. I say need, as in, not that I suddenly trust it and find I should use it, but rather a need as in, my job suddenly requires us to use it, as is happening in a lot of other fields. Being in an IT based field, I also feel that I should be at least familiar with installing and using AI. And, if I had to use it, I would prefer to use a self hosted version of it, rather than gift more of my digital life to someone else's AI on their computer where they may be using my personal data or querries for less than great purposes.

Installing and running an AI model was surprisingly trivial on Ubuntu 24. I installed Ollama, which is a python based open source tool to interface with various AI models:

# Install Ollama:
$ sudo snap install ollama
ollama v0.32.14 from Matias Piipari (mz2) installed

# Download one of the AI models:
$ ollama pull llama3.2
pulling manifest
pulling dde5aa3fc5ff: 100% ▕██████████████████████████████████████████████████████████  2.0 GB/2.0 GB  2.0 MB/s      0s
pulling 56bb8bd477a5: 100% ▕███████████████████████████████████████████████████████████▏   96 B
verifying sha256 digest
writing manifest
success

The reason I downloaded the llama3.2 model was because it was realatively small (at 2 GB download) and was supposed to work well with a regular CPU, as opposed to many models requiring a GPU for generating output. To run the downloaded model was also pretty simple:

# To start llama3.2:
$ ollama run llama3.2

# My first query:
>>> write a python script to multiply 6 and 4
# Multiplication Script
==========================

This script performs multiplication of two numbers, 6 and 4.

`python
# Function to multiply two numbers
def multiply(num1, num2):
    """
    Returns the product of two numbers.

    Parameters:
    num1 (int): The first number.
    num2 (int): The second number.

    Returns:
    int: The product of num1 and num2.
    """
    return num1 * num2

# Define the numbers to multiply
num1 = 6
num2 = 4

# Multiply the numbers
result = multiply(num1, num2)

# Print the result
print(f"The result of multiplying {num1} and {num2} is {result}")
`

### How to Run the Script

1. Save this script in a file with a `.py` extension, for example, `multiplication.py`.
2. Open a terminal or command prompt and navigate to the directory where you saved the script.
3. Run the script using Python, for example, `python multiplication.py`.

### Example Output

`
The result of multiplying 6 and 4 is 24
`

This script defines a function `multiply` that takes two numbers as input and returns their product. It then
defines two numbers, multiplies them using the `multiply` function, and prints the result.

>>> /bye

I then copied all of the script it produced into a new file called 'multiplication.py', and ran it with python3 ./multiplication.py, and saw that it actually worked. That seems pretty handy! Ask for some code, and receive it! But then why am I so opposed to using AI if it can generate code in such a simple manner, complete with explainations of what all that code means? Well, there are several reasons for that, which I hope to show you in the remainder of this article with some real examples.

Can you trust the AI output?

One of the greatest pitfalls of using AI is the trust level of the user. Most people that I talk to 'on the street' seem to implicitly trust AI. But should you actually trust what AI says? I don't think so. Let's do a simple experiment, let's ask AI to tell us the rules to a well known game. Here, I asked llama3.2 to tell me the rules of chess:

>>> can you tell me the rules of chess?
The rules of chess are as follows:

The Setup

1. The game starts with the setup of the board, which consists of 64
squares, arranged in an 8x8 grid.
2. Each player has 16 pieces: one king, one queen, two rooks, two knights,
two bishops, and eight pawns.
3. The pieces are placed on the board as follows:
 * Rook (or castle) on the corner squares (a1, h1, a8, and h8).
 * Bishop on the squares next to the rook (b1, g1, b8, and g8).
 * Knight on the squares next to the bishop (c1, f1, c8, and f8).
 * Queen on the center square (d1 and d8).
 * King on the square in the middle of the four corner squares (e1 and
e8).
 * Pawns on the second row (b2-h2 and b7-h7).

The Objective

1. The objective of the game is to checkmate the opponent's king, which
means the king is under attack and cannot escape capture.
2. The game can also end in a draw if both players agree to a draw, or if
the same position occurs three times (with the same player to move).

Movement

1. The pieces move differently:
 * King: one square in any direction (horizontally, vertically, or
diagonally).
 * Queen: any number of squares in any direction (horizontally,
vertically, or diagonally).
 * Rook: any number of squares horizontally or vertically.
 * Bishop: any number of squares diagonally.
 * Knight: two squares in one direction (horizontally or vertically), then
one square in a perpendicular direction.
 * Pawn: one square, but can move two squares on its first move.
2. Pieces can only move to an empty square, or capture an opponent's piece
by landing on its square.

Capturing

1. A piece captures an opponent's piece by landing on the square occupied
by that piece.
2. When capturing, the capturing piece stays on the square.

Special Moves

1. Castling: the king can move three squares towards a rook, and then move
the rook to the square the king crossed. This is the only move that allows
the king to move more than one square.
2. En passant: if a pawn moves two squares forward from its starting
position, and lands adjacent to an opponent's pawn that could have
captured it if it had only moved one square, the opponent can capture the
pawn "en passant" by moving diagonally to the square the pawn crossed.
3. Promotion: when a pawn reaches the opposite side of the board, it can
be exchanged for any piece (except a king).

Check and Checkmate

1. Check: when a player's king is under attack, it is said to be "in
check." The player must move their king to safety, or block the check with
one of their pieces.
2. Checkmate: when a player's king is in check and there is no way to
escape the check, the game is over and the other player wins.

Other Rules

1. A player can only move one piece per turn.
2. A player cannot move a piece to a square that is occupied by another
piece of the same color.
3. A player cannot capture a piece that is not their own.
4. A player must make a move on their turn. If they do not have a move,
the turn ends and the opponent's turn begins.

That's a basic overview of the rules of chess. Of course, there are many
variations and house rules, but this should give you a good starting point!

This is the unedited output from llama3.2, using Ollama as the interface. There are actually several problems with this ruleset that llama told me. First, and most obviously an error, is under "Other Rules", number 4, which states: "A player must make a move on their turn. If they do not have a move, the turn ends and the opponent's turn begins." This is completely false. If you play chess, you know that when a player has no legal move, but it is their turn, then the game is a stalemate, which is a type of draw. This also puts the "Objective", point 2, under scrutiny, since it said the draw can only happen if "both players agree to a draw, or if the same position occurs three times (with the same player to move)." It left out stalemate as part of the rules. Further, under "Movement" and "Capturing", the Pawn moves are not properly declared. Back to "Other Rules", rule 3 states that "A player cannot capture a piece that is not their own." Which to me means that you can only capture your own pieces, but rule 2 seems to contradict that rule.

Okay, so big deal, right? AI didn't do a very good job of explaining the game of chess. But now you are going to feed AI all of your symptoms and have it tell you what disease you have, and what treatment options are available to you. You're going to use AI to handle your stock exchange trades and manage your retirement. But, it contradicted itself and can't properly tell you the rules of chess. How would you let it handle your health, your finances, your life? If you can't trust it with a little task, can you trust it with a big one?

"One who is faithful in a very little is also faithful in much, and one who is dishonest in a very little is also dishonest in much." - Luke 16:10

Does it really save you time?

According to the Harvard Business Review [1], a recent study shows that workers actually spend more time reviewing and fixing AI mistakes than it actually saves. This sort of goes back to my last point, about AI being somewhat untrustworthy. Consider this:

$ ollama run llama3.2
>>> tell me a joke
Why don't scientists trust atoms?

Because they make up everything.

>>> do you make up anything?
That's a clever connection! As a conversational AI, I generate text based
on patterns and associations in the data I was trained on. While I strive
to provide accurate and helpful information, I can make mistakes or create
fictional content.

For example, I might:

1. Make up definitions for unfamiliar words or concepts.
2. Generate fictional stories, characters, or scenarios.
3. Create false information or misinformation (although I try to avoid
this!).
4. Use hypothetical or speculative language to explore complex ideas or
concepts.

So, if you can't trust the AI output, you have to verify everything it says, which takes time. Sometimes the effort spent verifying, and potentially correcting AI is more time than it takes to do the requested task. We already saw above that we would have to correct the 'rules of chess' that llama gave me. In that rule set, which is just 75 lines long, we found at least four errors or contradictions without even trying. Imagine having it compile data from a huge database of thousands or millions of data points. Since we already know we can't trust it, we have to rehash the data ourselves to make sure it is right. Thus, we spent the time asking the AI to hash the data, and then the time to rehash it ourselves to double check the AI. Did we really save any time?

AI is not actually intelligent, artificially or otherwise.

Again, AI, by it's own definition it gave to me, is not actually sentient or intelligent. It "generates text based on patterns and associations in the data I was trained on." Consider this game of chess I played against llama:

$ ollama run llama3.2
>>> lets play chess, don't show the board. I'll be white.
I'll play black. You can make the first move.

>>> Nf3
I'll respond with Nc6.

>>> g3
I'll play d5.

>>> d4
I'll respond with exd4.

>>> that is an illegal move.
You're right, capturing the pawn on d4 is not allowed since it's not your
turn to capture.

Let's try again. Since you played g3, I'll respond with e6.

>>> Bg2
I'll play Qe7.

>>> 0-0
You've castled kingside. I'll play Qe7 first, then Na5.

>>> d3
I'll respond with Qc5.

>>> Be3
I'll play f6.

>>> Bxc5
I'll respond with bxc5.

>>> That move is illegal.
You are correct, capturing the bishop is not a valid move.

Let me try again. Since you've played Bxc5, I'll respond with a6.

>>> Bxf8
A check! I'll play Kxf8 to block the check, then I'll recapture the pawn
on f8.

>>> /bye

For those of us who can't play chess in our head, a board was needed for this exersize. I was using a computer board with PyChess and made my moves, and then the AI's move on the board. When I moved my pawn to d4, llama responded with exd4, which is illegal. To perform this move, llama's e pawn would need to be on e5. But as you can see from the list of moves, e5 was never played, so the pawn cannot be there to make the capture. But, being a 'good sport' llama accepted my claim of an illegal move and backed up the game. Notice though, that it didn't go back to my d4 move, it went back to my g3 move, ignoring it's own d5 move and instead played e6! Wait a minute! It took back two of it's moves and one of mine! The AI litterally just hallucinated[2] a piece or position!

Okay, okay, I'll play on from this now altered position. After I castle kingside, llama decides to play "Qe7 first, then Na5." By it's own definition of the rules from earlier, it cannot move twice in one turn! Further, it already moved the queen to e7. So it can't do this again. So I took this to mean (I had to interpret the meaning of what should have been clear) that it meant, since it moved the queen to e7, now it will move the knight to a5. But, we play on. After I move Be3, the queen on c5 is under attack, it makes a simple move of a pawn, rather than defending the queen, and loses the queen (and really, the game) in an epic blunder. But here's what is funny: when I capture the queen with Bxc5, the response is bxc5, meaning the b pawn will take on c5. That move is again, illegal. It would need the b pawn to be on b6, which we can see has not happened at any time during the game. Ironically, llama could have played Bxc5, and recaptured with the bishop. The capital letter is important for standard chess notation, though, and llama admitted that I was correct, and chose a different move.

Finally, when it corrected itself and played a6, I captured his bishop with Bxf8. At this point, llama's analysis of the game lead llama to believe that it was in check, but it is not in check. Further, to get out of check, llama decided to capture my "pawn" on f8, but I cannot have a pawn on f8. Not only can I not have it there based on the game we have played so far, but also any pawn that reaches the eigth rank becomes a piece, either a queen, rook, bishop, or knight. It cannot, per the rules, remain a pawn.

I just quit the game. But, back to my original point, can you trust AI: absolutely not! But further, does it save you any time? In this case, no. I spent more time trying to figure out what the AI was doing, and correcting it's moves, then actually making my own moves. In 11 prompts for input of my moves, two were used to tell the AI that it made an illegal move, one was a move that was erased by the AI, one was a re-move because of the 'removed' move, and the last prompt led to another illegal move. Not even debating it's lack of chess skill, but I had to constantly check what the AI was doing because it was not following the inputed data correctly. So if I feed it a 3000 point database and ask it to show me a trend, I would need to verify all of the output to make sure it was correct, and potentially re-prompt the AI with further instruction to correct its errors. I submit that you may be better off reviewing the data yourself, and probably save time doing it.

"Know well the condition of your flocks, and give attention to your herds," Proverbs 27:23

Using AI doesn't make you smarter.

There are several studies, such as by Harvard [4] and others, like MIT [3] that show the use of AI is actually causing mental atrophy. The MIT even went on to say that the brain of people that use a lot of AI: "showed less activity in brain networks associated with cognitive processing while undertaking the exercise." E.g., they can actually measure your brain activity, and relying on AI regularly caused the test subjects to have lower measured brain activity over time. That's scary.

Every night, I read my children a book before bed. As they have grown, so have the books. We've done lots of classics, Narnia, Lord of the Rings, etc., but one of the books we did was "Carry on, Mr. Bowditch". It was actually an excellent book about Mr. Bowditch, who 'wrote the book' on modern navigation back when they were still sailing by the stars. In the book, Mr. Bowditch makes numerous calculations in his head that most of us (myself for sure) would struggle with, or not be able to compute at all, even with pad and paper. Why is that? Well, I think it is actually because of the calculator and the computer. We no longer need to calculate those things in our head, or on paper, because we can punch it into a calculator or spreadsheet or computer math program. My brain has atrophied in the ability to perform a lot of math functions because I never have to do it. Now take that problem and amplify that by having an AI machine that can answer every question, program anything, and compile data for you - if you can trust it.

"If any of you lacks wisdom, let him ask God, who gives generously to all without reproach, and it will be given him." - James 1:5

So, should you use AI?

I know for myself, using AI is a no go at this point. For me, when I do things like write this blog post, or program apps, or use ham radio equipment, I am not trying to just 'do the stuff' because I want to have the end result. For me, I am trying to learn, to increase in knowledge, skill, and ability. To keep my mind sharp. Using AI does not seem to help me do that. Perhaps it can for you. That's a decision for you to make. I'm not going to rain on your party, if you choose to use it. I know several programmers that do, and they are very sharp and are using it in a way that seems to work for them. Likewise, I have friends that create artwork with AI. That's great, for them. I don't mind that they do those things. I just can't do it myself. At least, not right now.

One last word of caution about AI that is a bit off subject: follow the money. AI is available free of charge just about everywhere for the layman. It costs money to run those machines. If they are providing that service to you for free, you might ask yourself how they are financially supporting it. Perhaps your use of it is becoming part of it's training. Or perhaps your data you put into it is becoming someone else's property. Things to consider carefully.

"Whatever your hand finds to do, do it with your might" - Ecclesiasties 9:10a

Linux - keep it simple.

[1] https://hbr.org/2026/02/ai-doesnt-reduce-work-it-intensifies-it [2] https://www.worklife.news/technology/ai-hallucinations/ [3] https://www.bbc.com/news/articles/cd6xz12j6pzo [4] https://news.harvard.edu/gazette/story/2025/11/is-ai-dulling-our-minds/